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The Problem of the Effective Definition of "random noguonco " 

Mathematicians have always had difficulty in coming to agree- 
ment over what is meant by "randomness' 1 * In order to agree on a 
formal model for a "random process", we have to agree on what intui- 
tive aspects of the matter we want to build into our system* The 
most prominent point of agreement is that the process should be un- 
predictable, but this is in itself a very small beginning* The solu- 
tion that has become conventional in modern mathematics is based on 
the notion of "random Variable", a highly technical notion in which 
the basic process is represented as a certain kind of infinite func- 
tion-space* This space contains all possible observed behavior 
sequences together with a "measure" structure which enables one to 
calculate the relative frequency of certain ("measurable") complex 
events* "Event" here usually refers to a whole class of behaviors* 

In this generally accepted theory* the individual "random 
sequence" plays a very small role, because its Individual measure 
or probability is zero, in non-trivial cases* Thus, if one flips 
a fair coin, the probability of any particular sequence 

,1s evidently zero# Furthermore, for this 

situation all individual sequences are equally likely* Yet the par - 

^ -• - " 

toioular sequence ' is "obviously" non-randoml 

If we can say that a particular sequence is not random, shouldn f t 

we be able to say that some other sequence is random? Until one 

becomes ever so sophisticated, this feeling lingers • But how could 

one characterize a single random sequence; On* would have to be 
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careful, for if the description is too specific, the sequence might 
lose the precious quality of controlled unpredictability. 

It would be very pleasing and useful, if one could set up a 
satisfactory theory in which it was meaningful to talk of an indi- 
vidual random sequence. It would be even more pleasing and useful 
if one could define this in some effective manner. 

It seems safe to say that we could not admit any computable 
sequenoes to the category of random sequences. For a computable 
sequenoe has the character that one needs only a finite amount of 
information in order to be able to make perfect predictions.** 

On the other hand, it might seem safe to say that a sequence 
should be considered random if there is no way to make a "better 
than chance" prediction abou t its next term, given the past behavior . 

The constraint that there be "no way to make a 'better than 
chance » prediction' 1 has the right spirit, but it is far too vague 
to be useful as a mathematical condition. 

We will formalize the notion is a certain "effective" way and 
then show that in at least that sense, there will exist computable 
sequenoes which sati sfy our definition of randomness . This result 
may be viewed in two ways. On the one hand it shows the futility 
of trying to make effective such a definition, if one's goal is to 
obtain an absolute notion of randomness. On the other hand, it does 
show that there are computable sequenoes so irregular that one can 
be constructed to defeat any such systematic attempt: that is, there 
are arbitrarily "disordered" yet effeotively-defined "pseudo-random" 
sequences. 

♦# Indeed, the reader unfamiliar with the definition of oomputability 
can take this as the definition. 

* The name of R. Von Misea' is associated with such attempts. See 
ref. [k]. 
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We will use a very crude measure of prediction ability* Let 
M be a (Turing) machine which computes some binary function of 
finite binary sequences • That is, given any finite string of f 3 
and l f s, M will produce a or a 1* Given some infinite sequence 
S(l), S(2), * * * we can have M produce , for each term 5(t), a 
digit M(t+1) which we will interpret as a "guess" about what will 
be the next term S(t+1) of the sequence S* We keep a "cumulative 
score 11 G(t) for M , adding 1 to it for each correct gness and 
subtracting 1 for each lnoorrect guess* Ifi M can consistently do 
" better than chance* then we might find, in the long run, that 

11m ^r^ - a > 
t-*» z 

(assuming that the limit exists at all)*) 

Clearly, for any particular machine M we could find computable 
sequences S(t) which would i) force the limit to 0, or ii) force 
the limit to 1 unity, and ill) prevent the relative score from 
converging to any limit at all* On the other hand, given first any 
computable sequence we could find a machine M which perfectly pre- 
dicts the sequence (by using the same machine that generates the 
sequence) • The problem vaKat; will concern us is that of the case in 
which a countable collection of machines M, are all simultaneously 
trying to predict the sequence* Can we find a sequence which will 
be able to "baffle" all of them simultaneously? The answer is that 
if the countable class »M. is effectively defined then there exists 

an effectively defined sequence which will Baffle them all/ 

* Indeed, we would obtain a satisfactory notion of randomness if we 
accept a sequence as random if and only if it baffles all effective 
prediction methods* But since the latter cannot be effectively 
enumerated, we reject this as an unrealistic definition* 
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Let us accept for the moment the definition that if a m 

for a particular machine and sequence, that machine has boon fooled 

by that sequence • Now suppose that a class >Mj, of Turing mocMnon 

has been defined effectively « that is, there is given a ft ma3ter" 

machine M which, given the "integer f j f and a sequence S(0),S(1),».., 

S(t), computes what M. would give for that same sequence • Suppose 

further (but informally, now) that this collection has been so chosen 

ad to include all known tests for statistical regularity, so that 

there is at least some basis for thinking of the machines as actually 

trying to make predictions* 

It was shown by Minsky and Silver (Ref • 1) that under these 
f 
conditions there always exists a computable sequence for which 

a «? for every one of the machines M ' # The question of whether 

this' remains true for machines which don't have to guess every time 

I 
was 1 unanswered in [1] • It is felt that this case is important, be- 

oause it allows more flexible "statistician machines" which can wait 

for certain special events and only then guess • Thus, a machine 

might wait until there was a run of 1,000,000 consecutive f s, arixi 

then guess that the next term will be 0# This would be a sound 

policy in the real world, for if the process is really random, one 

will do no worse than chance $ while if the long run is due to a 

systematic defect in the experiment, that defeot is liable to remain 

operative for some further time* The theorem was extended to this 

more general case by Levin, 4 and the proof below is essentially that 

in his thesis [2] • 

This concept of guessing is now extended to allow the guessing 

machines to guess only some of the time* This seems to be a fairer 
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This means that we cannot use the simple score function described 
earlier , but must take the score relative to the number of actual 
guesses. 

i ' r We now state a number of definitions . The sequence to be pre- 
dicted is S(t) = S(l), S(2), . .. The sequence — or rather, its 
'1 

initial segments— are examined by a sequence of Turing machines M,. 

It is required that each of these machines compute a total function 

over the set of all finite sequences. That is, given a finite sequence 

(of f s and l f s) the machine will eventually halt, giving an output. 

The n fch guess of the machine M, is written M,(n) and is the value M. 

gives in response to the sequence [S(lj, . • .,S(n-l) ]. This value 

must be either 0, 1, or * (wMch signifies "no guess"). 

We keep a running score G.Ct) of the number of correct 

guesses minus the number of incorrect guesses. Let H,(t) be the number 

of times the machine has guessed rather than indicated 0. Then we will 

say that if either 

g (t) 

i 1 ^-^ = or if lim H.(t) < <*> , 

Z H.(t) t -* <~ J 

the machine M.(t) has failed to predict the sequence S(t). In the 
J 

first case the machine guessed correctly no more than half the time. 
In the second case the machine stopped guessing and we cannot give it 
credit for being lucky on a finite number of guesses. In all other 
cases we consider the machine to have made a significant prediction 
of S(t). 

Returning to the idea of random nets, one might hope to define 
a sequenoe as random if it can pass a certain family of tests. By 
using the Turing Machines NL, we restrict the tests to be effectively 
computable. We have required that the test values themselves be .._ . 
defined — that the machines terminate each computation. 
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One attempt at 3uch a definition would simply include all 

possible guessing machines in the test family. A sequence whixh 

passes all such tests will surely appear random. Clearly these could 

exist no such computable sequence, but as shown by Church[3]»there 

must exi3t (non-computable) such sequences. However, this definition 

cannot be made effective, because the set of tests itself — the 3et 

of Turing machines which compute functions defined over the whole 

input range cannot be effectively defined. Instead, we consider 

an arbitrary effectively enumerated sequence of testa the sequence 

{mJ of machines defined above. Our theorem is 

Given any effectively enumerated sequence of teat machines ML, 

there exists a computable sequence S(t) which baffles all the M,. 

Since the sequence [mX of machines is effectively enumerable, 

Jthere exists a single machine that computes the same function as M . 

when supplied with the Integer J. That is, there is a machine U such 

that Mj(t) «U(j,t). 

To show how the sequence S(t) is constructed, we begin with the 

special case in whfcch there is only a finite number, n, on machines. 

Assume that S has been computed for all t< t Q . Thon Ve can compute 

U(j,t) for all li j< n and l< t< t Q . For each t the guesses of the 

n machines form an n-vector V[t] « [U(l,t),U(2, t), ...,U(n,t)] . There 

are 3 possible values for V[t] . To compute S(t Q ) we count the number 

of values of t for which V[t]=V[t Q ] and t<t Q . Assign the value or 1 

*1P- jS.Cfcp). A a this number i3 even or odd. 

To illustrate, suppose there are only two machines, and they 

have just made their 10 guess: 



♦But not for the optional guessing case. If we remove the con- 
dition in the theorem below, that the machines be effectively 
enumerable, we .obtain Church's reaulfc, since the construction 
carries through (non- effectively) without this condition. 
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The vector V [10] is (1,0). .Since this identical vector has already 
oocurred exactly twice before (V[10]s V[7] ■ V[l]), and two is even, 
the value of S(10) should be 0. 

Now consider any vector type, for instance the vector type of 
V[l], V[7], and V[10]. The value of S(t) alternates strictly on 
the subsequenoe 1, tf, 10, • ,., If the correct-incorrect score G,(t) 
were kept only on this subsequence of t, then it could never become 
greater than 1. But there are at most 3 n such subsequences, corres- 
ponding to the 3 vector types when there are n maohines. Therefore: 

2) G.j(t) < * n 

If for a given machine M,, 11m H,(t) « » this Implies: 

J t-*» ' J . ■ • 

cut) 

Therefore M, has failed to predict S(t). 

To extend this technique to Infinite sets of machines, we 
cannot consider infinite vectors, but we oan Increase the number of 
machines under consideration so as to include each of them eventu- 
ally, 

▲ value of S(t) obtained by applying the above recursive scheme 
to the first J machines will be called S'(j,t)* The number J will 
be called the depth of the computation. In general, the depth tends 
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to Increase with Increasing t# By deciding on the correct depth 
for each t, we can oompute S(t) by setting S(t) « S f (j#t) # 

The rule for computing the depth Is now given ♦ Let t Q be fixed 
and consider the sequence S f (l,t ), S f (2,t Q ), . ♦. • In computing 
S^jftrj) a count was made on the number of earlier vectors that are 
identical to V[t Q ] * (S» (1, t Q ) , • . .,S» ( J f t Q ) ) . If this frequency 
count is greater than 3 % then the computation of S f (J,t Q ) will be 
called excessive* The correct depth (J) is now given as the smallest 
value of j for which S*( j,t Q ) is not excessive ♦ 

Having stated the algorithm, we must pow prove that It works 
by establishing the following facts: 

1* We shall consider a particular machine M and prove that 
It cannot predict the sequence S(t) # If M n makes only finitely 
many guesses , then there is nothing to prove • We assume that It 
makes Infinitely many guesses for the rest of this proof • 

2# For each guess M (t), S(t) was made at a particular level 
j, and for a particular vector type V[t] » (M^( t),#**,M,(t) ) • At 
level j there are 3 J . different vector types • At most 3 J computa- 
tions are made for each vector type before it becomes excessive • 
Therefore , at most 3 computations are made at each level. 

This Is to say, that if we rewrite S(t) to Indicate the level 
of each computation by writing S»(J lf l) f S*(J 2 ,2), # .», then each 
Integer J will occur at most 3 times • 

3 * There are an infinite number of vilues of t for which 
M (t) ^ ][) # Since only a finite number of them are made at any one 
level and sinoe no level oan be skipped, M guesses at all levels • 

In faot, we can establish a minimum number of guesses atclevel j 



by the following reasoning: 

a. M guesses at lca3t one* at lcvt?l ,1+1 „ This vr^nnu »•!;-. f- 
for soma t, S»(j,t) is .excessive for a vector V[t] « C< (^) j. « «■ -M , (t) 'J 
such that M n (t) » or i. I^et T(j) be th« l«.*asfc auch fc. Tivn thr 
vector Vfcl has occurred 3 2J times before time T(j). No to thufc the 

sequence {T(j)j, forms a strictly monotonic unbounded sequence. 

2^1 i} 
b After the vector V[t] occured 3 w "' ' timr?s the level 

(j-1) became excessive. Therefore 3 «* - 3 2 *J~ 1 ' = -§3 2 ^ guesses, 

at least, have been made at level j. 

4, We now wish to estimate ' ' ^ - ' •■ ' , Let T be the tim ■? such 

H(t) 
that for t>T, all guesses are made at levsd n or higher o (We are 

considering the score for machine M •) For all levels J<n tha 

machine may do very wello But even if all its guesses are correct, 

there are only finitely many of them To be precise, these can 

add at most T to the score Q £'5], 

Starting with level n, M guesses right only half the time 

on each vector type At level j there are 3^ vector types and at 

least -g3 guesses are made.. (See argument 3b.) Hence G(t) can 

change by at most 3 J at level J, while H(t) must increase by at 

least |3 2,I o 

5. Now for each t find the J - j(t) such that T(j) * fc < T(j+i); 
that is, the largest J such that T(j) ^ t. We can then bound G(t) by 

G(t) s t + il£) 3 J , 

for no guesses have been made at levels higher than J(t). We 

can set a lower bound on H(t) by the argument in (4) above: 

o J(t)-1 0i 

Hitj^n 3 2J . 



■ i '!«»■ m ■■><—■ 



s lim 
t —a 



--io 



Now, by (3), as t -> < 3°, J(fc) -*oo >30 we have 

"a(t) , „ T + £3 J 

— *- *■ * lim j=n 

H(t) j(t) ~~ F~5(l)ri — - ° 

9 ^ 3 2J 

Now if we replace all expressions of the form M.(t) by the 
corresponding U(j,t), the construction becomes effective, S(fc) is 
a computable sequence, and the theorem is proved. 
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